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ABSTRACT 

This study explored questions about whether there is a positive or 
negative relationship between student computer use and achievement, and whether the 
result vary by the amount of school or home computer use. Findings from the Idaho 
Statewide Testing Program for 31,000 students from more than 300 schools show that in 
general there is an inverse relationship between in-school computer use and student 
achievement. However, there is an overall positive relationship between student 
achievement and computer proficiency (reported capability with a variety of software) . 
It is important to note that the student software capability is related to use both at 
school and at home. Because of the differing interpretations that are possible and the 
importance of the topic, caution is urged. Researchers and policy makers must think 
carefully when interpreting correlational results between achievement and student 
technology use measures, regardless of whether the direction of the proposed 
relationship is positive or negative. (Author/SLD) 
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ABSTRACT 

This paper explores questions about whether there is a positive or negative relationship between student 
computer use and achievement, and whether results vary by the amount of school or home computer use. 
We find that generally there is an inverse relationship between in-school computer use and student 
achievement. However, there is a positive overall relationship between student achievement and computer 
proficiency, i.e., reported capability with a variety of software. Importantly, the student software capability is 
related to use both at school and at home. Because of the differing interpretations that are possible and 
the importance of the topic, caution is urged. Researchers and policy makers must think carefully when 
interpreting correlational results between achievement and student technology use measures, regardless of 
whether the direction of the proposed relationship is positive or negative. 



PURPOSE 

In today’s climate of educational accountability, considerable attention is focused on students’ academic 
achievement and the school environment necessary to develop and support achievement, An up-to-date 
technological infrastructure is generally considered a key part of an effective school environment. Special 
attention has been focused on the “digital divide” separating disadvantaged urban and rural students from 
more advantaged suburban students. Government funding has been focused on urban and rural schools in 
order to infuse technology In these schools and provide disadvantaged students with technological 
opportunities equal to those found In suburban schools, Such an approach assumes that it is computer use 
at school that is associated with academic achievement. Our analyses have led us to question, however, 
whether this is indeed the case. 



The purpose of this paper is to explore the relationships between student computer use at school, 
computer use at home, and academic achievement. A few studies have indicated that there are 
achievement advantages for computer using students, particularly those who have access to home 
computing opportunities. For example. Green (1998) reported greater achievement gains in language arts, 
reading and mathematics among S*! and 1 1*’ grade students who were more competent and frequent 
computer users. Most studies, however, have not distinguished the separate relationship home and school 
computer use may have with academic achievement. An exception is Weglinsky (1998), who found that 
home computer use was positively related to academic achievement, while an emphasis on in-school use 
was negatively related to academic achievement. This last finding, in conjunction with other research 
(Becker & Ravitz, 1998; Becker, Ravitz & Wong, 1999) suggests that there may be a “pedagogical divide” 
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at school where lower achieving students engage in more rudimentary computer uses devoted to drill and 
remediation. 

This paper seeks to address the follovwng research questions: 

1) Descriptives: How do schools and students differ in technology use and achievement, 
and what are major variables that need to be controlled to identify an independent effect of 
technology? 

2) Computers and Achievement: Are higher achieving students more proficient computer 
users? Is computer proficiency related to both raw achievement and achievement gains 
from year-to-year? 

3) School or Home: Which location of use is more closely associated with computer 
proficiency, raw achievement scores, and achievement gains from year-to-year? 

METHODS 

This study uses individual and school wide and student-level measures of student achievement based on 
the Iowa Test of Basic Skills and the Test of Academic Proficiency (ITBS / TAP). 

School Data Set. To address variations in technology presence and conditions at the school level we 
used the School Technology Inventory completed by school or district level administrators throughout the 
state. For a measure of school size we used the athletic categories used in Idaho for both 8*' and 1 1 
grade schools; these are based on the number of students in each school. 

Student Data Set. We obtained data from the statewide administration of the Iowa Tests of Basic Skills in 
Language Arts, Reading, and Mathematics (ITBS). The total number of students in the study is 31 ,000 
from over 300 schools. As part of the Idaho Statewide Testing Program, all 8*’ and 1 1*^ graders completed 
a 17 item self-report instrument describing their competency with educational computer use, their 
opportunities to use computers in school, and the frequency mVn which they used computers at school and 
at home. 

• Measuring Achievement: The ITBS scores for Reading, Language and Math from both 1999 and 
2000 were all highly correlated (r > .72). Achievement z-scores were assigned after splitting the file by 
student gender and grade (8*’ or 1 1 0’) because girls scored higher in language arts. 

• Measuring Achievement Gains. Because scores generally rose between 1 999 and 2000, we want to 
compare students’ gains. In addition, because higher scoring students and lower scoring students may 
gain at different rates, we want a standardized way of comparing gain scores. The standardized 
residual gain score indicates the gain in test scores relative to what would have been expected based 
on knowledge of the first year test scores. 
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• student Self-Ascribed Computer Proficiencies and Use: A subset of the 17 item instrument asked 
specifically about capabilities to perform tasks, such as word processing, spreadsheets, presentations, 
Internet and email. Students were asked to indicate (8, 4-7, 1-3, 0 hours) how much they used 
computers at school and at home. The index score had a reliability alpha of .75. 

• Statistical significance. Due to the large sample sizes, even inconsequential differences are 
statistically significant at p < .001 . Eta provides a better estimate of the differences in means between 
ordered groups of cases. 

FINDINGS 

To date we can report the follo\Aring findings from our work: 

Using percentage of student users as a measure, there is a negative relationship between use of 
computers by students at school and school wide achievement. This is because a greater 
percentage of students in smaller, lower performing schools, and a smaller percentage of students 
in larger, higher performing schools, use computers at school. 

There are substantial differences in student technology use and achievement by school size and grade. 
Most of Idaho’s schools are small and in rural areas, but more sfudenfs attend the few larger, urban 
schools. The rural, small schools have more computers per student and a larger proportion of students 
using computers in school, compared to urban, larger schools. Almost 1/3 of homeonly users are in the 5*' 
(wealthiest) MFI89 quintile, while over Va of the school-only users are h the 1«‘ (poorest) MFI89 quintile. It 
seems a few computers go a long way in small schools and not in large schools. While students in urban, 
large schools less often use computers at school, they more often use computers at home. 

Larger 8*' and 11*' grade schools had higher ITAP/TAP test scores in 1999 and 2000. Larger 11*' grade 
schools also gained more on the TAP from 1999 to 2000; however, for 8*' grade schools, there is some 
evidence that it was the smaller schools that gained more on ITAP from 1999 to 2000 (Ravitz & 
Mergendoller, 2002). 

Looking at Table 1 , it is apparent that family income is related to school size. Patterns of computer use at 
home generally follow school size (and income) patterns. Patterns of school achievement are positively 
related to home computer use and family income and inversely related to school computer use. This 
relationship generally persists, even controlling for school size. 

The same pattern is found when one compares school achievement and income, or school achievement 
and home computer use. Larger schools enroll higher achieving students at both the 8*' and 1 1*' grades, 
across all three subject areas (ibid). Note that this reflects test scores at one point in time (October 2000), 
and does not address the issue of whether there were differential achievement gains between 1999 and 
2000 in larger or smaller schools. 

Overall 2000 school achievement is reported on Table 1 using school-level standardized z-scores derived 
from an aggregate index combining 2000 mathematics, language arts, and reading scores for the students 
in each school. Standardized scores remove the effect of the mean and standard deviation, so the 
“average” score is 0.00 and the standard deviation is 1.00. Looking at schools containing the 8*' grade and 




Ravitz, Mergendoller & Rush (2002, April). AERA, New Orleans 



4 - 



3 



schools containing the 11'^' grade, we find a consistent pattern of the smallest schools exhibiting a negative 


z-score. The z-score then increases (e.g.. 


achievement scores rise) in relationship to the size of the school. 




Table 1. Location of Computer Use, Achievement, and Income Data, 








by School Grade and Size 
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Overall 2000 
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Average % of 


number of 


school 


Median 






students who 


students who 


school 


achievement 


family 




School size 


use 


use 


computers 


(standardized 


income 


Grad 


(sports 


computers at 


computers 


per 10 


z-score) 


(1990) in 
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categories) 


school 


at home 


students 


within grade 


thousands 


8 


<150 


74% 


62% 


4.1 


-0.48 


27.0 




150-349 


79 


73 


3.0 


0.03 


25.6 




350-799 


67 


78 


1.9 


0.19 


28.2 




800-1249 


53 


83 


1.9 


0.44 


33.7 




All 8*’ grades 


72 


73 


2.8 


0.00 


27.3 


11 


<150 


81 


72 


5.0 


-0.79 


27.3 




150-349 


79 


79 


3.3 


0.01 


25.8 




350-799 


73 


80 


2.2 


0.35 


26.6 




800-1249 


61 


84 


2.2 


0.55 


30.0 




1250+ 


50 


85 


1.7 


0.67 


31.0 




All 11*1 grades 


73 


79 


3.2 


0.00 


27.2 


All 


<150 


77 


67 


4.6 


-0.64 


27.2 




150-349 


79 


76 


3.1 


0.02 


25.7 




350-799 


70 


79 


2.0 


0.25 


27.5 




800-1249 


58 


83 


2.1 


0.51 


31.3 




1250+ 


50 


85 


1.7 


0.67 


31.0 



All schools 73 76 3.0 0.00 27.2 

Note : Standard deviation for PCs per 10 students = .22. Standard deviation of median family income, in thousands = 4.3. 



It is evident that one must control for school size and/or income to understand accurately the relationship 
between technology use and student achievement. It may be the case that /lome computer use has a 
greater impact on student learning in Idaho’s larger school districts, and sc/ioo/ computer use is more 
important in smaller, rural schools. 

Similarly, students who score better on standardized achievement tests are those who use 
computers more often at home, and less at school. 

Replicating the above findings at the student level, students who score better on standardized achievement 
tests are those who use computers more often at home, and less at school. This suggests again that home 
use, not school use is associated with greater achievement (Table 2). 




Ravitz, Mergendoller & Rush (2002, April). AERA, New Orleans 

5 



4 



Table 2. Mean Achievement (ITBS) Z-Scores by Grade, Gender, and Computer Use Location: 
Higher achieving students use computers less at school and more at home. 



Gender 


GRADE 


Computer Use at School per week 




0 

Hours 


1to3 

Hours 


4to7 

Hours 


8 or More 
Hours 


Total 


Female 


8 


.04 


-.05 


.08 


-.23 


.00 


Male 


8 


.04 


-.01 


.08 


-.28 


.00 


Female 


11 


.16 


-.08 


-.07 


-.17 


.00 


Male 


11 


.09 


-.04 


.01 


-.04 


.00 




Total 


.08 


-.04 


.03 


-.19 


.00 






Computer Use at Home per week 




Female 


8 


-.38 


.02 


.22 


.09 


.00 


Male 


8 


-.37 


.00 


.16 


.21 


.00 


Female 


11 


-.44 


.00 


.15 


.32 


.00 


Male 


11 


-.42 


-.08 


.15 


.35 


.00 




Total 


-.40 


-.01 


.17 


.24 


.01 


Note: Cells show mean z-score for row. Lowest Row or Column N > 4900, Lowest Cell n > 228.| 



Male students in 8®' grade who report using computers at school for 8 + hours per week score on average 
one-quarter standard deviation lower on standardized tests than do 8'^’ grade boys as a whole (Table 2). 
This pattern also seen for 8»' grade and 1 1 »' grade girls, but is not seen for 1 1»' grade boys, a finding that 
will be pursued in future analyses. 

At home, more use is associated with higher scores. The only exception is 8»' grade girls; moderate use at 
home (4-7 hours/week) is reported by the higher scorers. This is because a greater proportion of students 
in large schools do not have access to computers at school, but they do have access at home. 

Although we admit to having a very weak measure of SES (1989 mean family incomes for the 
school zip code) the relationships shown seem to operate independently of SES. 

Importantly, it is not just in high income schools where home computer users score higher on tests and 
school computer users score lower. More extensive users at home score higher on standardized tests and 
more extensive users at school score lower on standardized tests, even when we split the file by SES 
quintiles (Table 3). 

If the relationships between home use, school use, and test scores that have been shown were solely a 
result of SES, then we might expect the relationships to disappear when we control for SES by splitting the 
file. High SES students using computers at school would not exhibit lower scores; low SES using students 
using computers at home would not score higher than the group as a whole. 

Although we cannot rule out the possibility that there are still student-level differences based on family 
income within these quintiles, the relationships reported above generally appear to hold true even when we 
control for the SES of the school. 
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Table 3. Achievement and proficiencies by amount of home and school use, by SES 



MFI89 

Quintiles 


Hours of 
Computer 
Use 

Per Week 


Test 
Scores 
by Use 
at Home 


Proficiency 
scores by 
Use at 
Home 


% Users at 
Home 
(within 
Quintile) 


Test 

Scores by 
Use at 
School 


Proficiency 
Scores by 
use at 
School 


% Users 
at School 
(within 
Quintile) 


1st Quintile 


0 


-.49 


1.74 


23 


-.15 


1.91 


25 


Low income 


1 to 3 


-.19 


1.99 


35 


-.21 


2.07 


50 




4 to 7 


-.01 


2.21 


24 


-.09 


2.19 


19 




8 or More 


.10 


2.39 


17 


-.24 


2.34 


6 






(-0.17) 


(2.07) 


(100) 


(-0.17) 


(2.07) 


(100) 


5th Quintile 


0 


-.26 


1.61 


15 


.22 


1.95 


48 


High income 


1 to 3 


.13 


1.96 


40 


.12 


2.12 


34 




4 to 7 


.34 


2.20 


26 


.16 


2.24 


13 




8 or More 


.34 


2.44 


19 


-.12 


2.25 


5 






(0.17) 


(2.06) 


(100) 


(0.17) 


(2.06) 


(100) 


Total 


0 


-.40 


1.68 


19 


.08 


1.93 


35 




1 to 3 


-.01 


1.98 


37 


-.05 


2.09 


43 




4 to 7 


.17 


2.20 


26 


.03 


2.19 


16 




8 or More 


.24 


2.40 


18 


-.20 


2.30 


6 




Total 


(.00) 


(2.06) 


(100) 


(.00) 


(2.06) 


(100) 



Note : Similar patterns appear for the other quintiles. 



In summary, findings about how achievement varies according to school and home use are consistent; they 
occur somewhat independent of SES. 

Analyzing Student Software Capability instead of Home vs. School Use 

There is another conceptualization of technology use in schools that emphasizes how home use and 
school use might support each other to influence learning. One could argue that our question should not 
be whether the isolated school use contributes to achievement but whether a student’s overall experience 
with technology does. One could argue that capability with technology is more important than location of 
use and that limiting studies to classroom use and removing the effect of home use, is counter productive 
to understanding the real influence of technology. 

Besides conducting separate analyses of school and home use there are other ways to control for home 
and community effects. The first involves looking at within-school differences in proficiency and 
achievement, the second concerns looking at residual test score gains. We might decide to focus on 
student software capability, a measure that is related to student software use at home and at school. Doing 
so, we uncover findings that point to a positive relationship between technology proficiencies and student 
achievement. 
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Within schools, students who have higher software capability not only score higher on tests but 
they also gained more, on average, from 1999 to 2000 

Comparing student level data and looking at individuals within schools (our best available control) - 
students who have higher software capability not only score higher on tests but they also gained more, on 
average, from 1999 to 2000. These findings are, of course, statistically significant, p < .001 , effect size = 
.35 for 2000 achievement; effect size = .18 for year-to-year gains. 

Students were characterized compared to others within their school as being either high or low in software 
capability; we then analyzed how their achievement scores differed from others within their school. Table 4 
shows that student overall achievement within their school is related to their level of software capability. 
Students who are characterized as having high computer capability scored higher on the combined 
measure of mathematics, reading and language arts achievement than students who had lower computer 
capability within their schools. 

Table 4. Student Within -School Achievement, by Software Capability 

Students within their school characterized as Mean z-score Effect 

having... on 2000 tests N Size 

Low software capability -.15 14650 

High software capability ^17 14657 .35 

Total .00 29307 

Note : The mean z-score on the 2000 tests are shown, not gains. Effect size is based on the 
standard deviation for the 2000 combined test score index, .91 . The difference is statistically 
significant, p < .001 . 

The above table shows that students who report more software skills scored higher on the 2000 tests than 
others within their school. The effect size for students with higher software capability is .35, or about one- 
third a standard deviation. These scores place high software capability students, on average, at the 57th 
percentile and low software capability students at the 44*1 percentile, for students within their schools. The 
next table addresses whether students with high software capability also gained more than others within 
their school, controlling for their 1999 scores. 

It is important to make a distinction between technology use as a predictor of overall “raw” achievement 
(relatively easy to show, tied to many other variables) and technology use as a predictor of achievement 
gains (more difficult to show, because the analysis controls for prior achievement and related variables). If 
we look at residual gains in test scores we, to some extent, control for any prior advantage that computer 
users might have had in terms of achievement. For other approaches to reporting test score changes see 
Russell (2000). 
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Within schools, residual gains on test scores were also related to their computer capability. Those 
characterized as having higher software capability gained more, on average, than others within 
their same school. 

Table 5. Student Within-School Test Score Gains, by Software Capability 

Students within their school characterized as Mean residual Effect 

having. ■■ test score gain N Size 

Low software capability -.05 11157 

High software capability ^06 11331 .18 

Total .00 22488 

Note : Effect size is computed based on the standard deviation for 1999 test score index, .60. This 

difference is statistically significant, p < .001 . 

The table shows the gain scores of students compared to others within their school based on their reported 
software capability. These scores place high software capability students within each school, on average, 
at the 52nd percentile and low software capability students at the 48®' percentile, in terms of gains. In this 
case, a small percent of this many students is a substantial difference 

Together, tables 4 and 5 demonstrate that, within schools, it is the students who have more software 
capability who scored higher on 2000 tests and who gained more on tests from 1999 to 2000, controlling for 
prior achievement. 

Discussion 

Without losing sight of the larger numbers of students involved in larger schools, students in smaller 
schools are less likely, using percents of students instead of counts, to have computers at home. Because 
smaller schools also scored lower on ITBSTTAP tests overall, this creates an inverse relationship between 
school-only computer use and achievement. On balance, higher performing students tend to use 
computers more at home than at school. While this may point to the importance of home computing for 
higher achieving students, it does not mean use at school produces lower test scores. 

Students from poorer communities tend to rely more on computers at school. Students from higher income 
communities use computers more at home. One could argue that the relationship between home use and 
test scores is entirely due to SES. However, this would not account for low income students with 
computers at home scoring higher than high income students with no computers at home. It would also not 
explain the substantial relationship between home use and test scores that appear, even when we control 
on SES. 

When we examine students’ software capability scores, which in effect allows us to combine the effect of 
computer use at home and at school, we see positive relationship to student test scores and test score 
gains. Given that home computer use seems to be more closely associated with achievement and 
computing proficiencies than is school use, even controlling on income, conclude for now that lack of use at 
home is probably a more substantial barrier to achievement and the development of computing 
proficiencies than lack of access at school. 

9 
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We must caution that higher performing students who use computers at home are likely benefiting from a 
combination of conditions, while low performing students probably see substantially less home use, but 
also perhaps ineffective school use, and less favorable learning conditions at school and home. Prior 
research has shown that lower achieving students in particular tend to use technologies that focus on skill 
or drill games with the objectives of remediation and mastery of student skills. Future analyses will include 
more attention to pedagogical differences among teachers and to differences in prior achievement within 
schools. 
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